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SPECIFICATION 

Information Processing Apparatus Provided with Input System Utilizing Stroboscope 

TECHNICAL FIELD 

5 The present invention relates to an information processing apparatus provided 

with an input system utilizing a stroboscope. More specifically, the present invention 
relates to an information processing apparatus that processes an image signal of an object 
irradiated by the stroboscope. Furthermore, the present invention relates to an 
entertainment apparatus such as a game apparatus, etc. provided with an input system 
10 utilizing a stroboscope. In addition, the present invention relates to a man-machine 
interface system provided with an input system utilizing a stroboscope. 

PRIOR ART 

The applicant of this invention proposes that in the Patent Literature 1, and so on 
15 sensory video game machines in which change in acceleration of a bat or a racket in a real 
space is detected by use of a piezoelectric buzzer so as to be used as a game input. In such 
the sensory video game machines, when the change in acceleration is equal to or more 
than a predetermined amount, it is determined that the game player operates (swings) an 
object (bat or racket in the above-described example) in the real space. 
20 Patent Literature 1 : Japanese Patent Laying-open No.200 1 - 1 0463 6 

Patent Literature 2: Japanese Patent Laying-open No.2002-231489 
Patent Literature 3: Japanese Patent Laying-open No. 7-141 101 
However, depending on the kind of the game, information indicative of which 
position, which velocity, or which direction the operation is performed as well as whether 
25 or not the object is performed, that is, whether or not the acceleration is applied to the 
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object may be required. In the input device disclosed in the Patent Literature 1, it is 
impossible to fulfill such the requirement. 

On the other hand, if imaging the object by utilizing the stroboscope disclosed in 
the Patent Literature 2, for example, by analyzing an image signal, the position and the 
velocity of the object as described above can be obtained. However, in the Patent 
Literature 2, the stroboscope is only disclosed, and the Patent Literature 2 does not 
provide a specific method of imaging an object by use of the stroboscope, and analyzing 
in real time an image signal obtained through it. 

It is noted that the Patent Literature 3 discloses that an object is extracted from an 
imaged image signal to obtain the position of the object, and the position information is 
used as an input of game apparatuses or computers. However, this method works well in 
a specific use environment, but it is very difficult to obtain accurate position information 
in a room at a general home where game machines are used. This is because all of the 
illuminations in the room, windows, objects in various colors, and moving objects except 
for the game player exert an influence as noise and disturbance on the detection accuracy. 
In order to accurately detect the position of the object with the influence of the noise and 
the disturbance reduced, a high-speed processor is necessary, and this is unrealistic for the 
information processing apparatus with restricted processing ability and at low costs, 

SUMMARY OF THE INVENTION 

Therefore, it is a primary object of the present invention to provide a novel 
information processing apparatus, an entertainment apparatus, and a man-machine 
interface system that are able to apply an input in real time to a computer or a game 
machine by use of a stroboscope. 

The present invention relates to an information processing apparatus provided 



with an input system utilizing a stroboscope, and comprises a stroboscope; an imaging 
means for imaging an object at a light-emission and at a non-light-emission of the 
stroboscope to output an image signal at light-emission and an image signal at non-light 
emission; a first means for calculating a part or all of information of a position, a size, a 
velocity, an acceleration, a moving path pattem of the object on the basis of differences 
between the plurality of image signals at light-emission and the plurality of image signals 
at non-light emission; and a second means for performing information processing on the 
basis of the information calculated by the first means. 

The above-described first means may include a determination means for 
determining whether or not the information is coincident with a predetermined condition. 

In one embodiment, the first means includes a valid input detecting means for 
detecting only valid information out of the information on the basis of the determination 
result by the determination means, and transmitting to the second means as the valid 
information being performed. 

Furthermore, the first means includes a distance calculating means for calculating 
a distance between the object and the imaging means from the information indicative of 
the size of the object. 

In addition, the first means includes an analyzing means for analyzing information 
obtained from a difference between the image signal at light-emission and the image 
signal at non-light emission to extract the shape of the object, and an angle calculating 
means for calculating an angle between the object and the imaging means from the shape. 

The analysis by the above-described analyzing means is for extracting 
predetermined two points within the object, and the calculation of the angle by the angle 
calculating means is for calculating an angle between a line segment between 
predetermined two points and a predetermined coordinate axis. 



In an embodiment, a time interval of the light-emission of the stroboscope is freely 
configurable. 

Furthermore, a length of the light-emission period and a length of the 
non-light-emission period of the stroboscope are freely configurable. 

In addition, an exposure period of the imaging means is freely configurable. 
In one embodiment, the object includes a reflective body. 

Furthermore, the stroboscope includes a light source for outputting a light having a 
specific wavelength range, and the imaging means is made to respond to only the specific 
wavelength range. 

In this case, the imaging means includes a filter for passing only the light with the 
specific wavelength range and an imaging device for imaging an image formed by the 
light passed through the filter. 

In another embodiment, the imaging means includes an imaging device for 
imaging only the image formed by the light having the specific wavelength range. 

Each of the first means and the second means described above is process to be 
processed by a single or a plurality of processors. 

Then, the image processing performed by such the second means is an 
entertainment process such as a game, etc. 

A man-machine interface system provided with an input system by utilizing a 
stroboscope comprises a stroboscope; an imaging means for outputting an image signal at 
a light-emission and an image signal at a non-light emission by imaging an object at a 
light-emission of the stroboscope and at a non-light-emission; a first means for 
calculating a part or all of information of a position, a size, a velocity, an acceleration, a 
moving path pattern on the basis of differences between the plurality of image signals at a 
light-emission and a plurality of image signals at a non-light emission; and a second 



means for performing image processing on the basis of the information calculated by the 
first means. 

According to the invention, by the stroboscope (42, 52: reference numerals of 
elements or components corresponding in the embodiments, and so forth), the object (14, 
94, 112) is brightly irradiated to heighten a contrast between the object and things except 
for the object as an imaging result, and therefore, it becomes easy to detect the object. 
Furthermore, the first means (52, S59, S129, Figure 35: S61, Figure 22, Figure 25, Figure 
33) calculates the differences between the plurality of image signals at light-emission and 
the plurality of image signals at non-light emission, and whereby, it becomes possible to 
detect the position, the size, the velocity, the moving path pattern of the object with 
accuracy and with simple image processing while reducing an influence of noise 
component due to a still image except for the object being a moving object, a fixed light 
source, and etc. On the basis of the information thus calculated, the second means (52, 
S63) performs the predetermined information processing. 

Then, by separating the information processing for calculating these information 
and the information processing at the application side, it is possible to make the 
information processing at the application side simple, and at a time of replacing the 
information processing at the application side with another processing, the processing for 
calculating the position, the size, the velocity, the acceleration, the moving path pattern of 
the object can be utilized without being changed. 

In a case of utiUzing the determination means, the determination means (52, S61, 
Figure 22, Figure 25, Figure 33) determines whether or not the position, the size, the 
velocity, the acceleration, the moving path pattern of the object are coincident with the 
predetermined condition, and in the information processing at the application side, with 
reference to the determination result, if the information is not coincident with the 



predetermined condition, the calculated information is not received, capable of 

performing simple application processing. 

More specifically, the valid input detecting means (52, Figure 22, Figure 25, 

Figure 33) included in the first means makes a choice of the information on the basis of 
5 the determination result by the determination means, and transmits only the valid input 

from the user to the information processing at the application side. Therefore, it becomes 

possible to perform simple application processing. 

In a case of utilizing the distance calculating means, the distance calculating 

means (52, SI 1 1, SI 13) included in the first means calculates the size of the object from 
10 the imaging result, and calculates the distance between the object and the imaging means 

on the basis of the calculated size information. Thus, from the two-dimensional inaaging 

result, the position, the velocity, the acceleration, and the moving path pattem in the 

three-dimensional space can be obtained. 

In a case that the first means includes the analyzing means (52, Figure 24: 
15 S 1 59-S 1 67) and the angle calculating means (52, S 1 69), by analyzing the shape of the 

object from the imaging resuh, the angle formed by the object projected on the 

two-dimensional image as the imaging result and the imaging means can be obtained. 
The analyzing means performs the analysis (SI 66) for extracting the two 

predetermined points within the object, and the angle calculation executed by the angle 
20 calculating means is for calculating the angle between the line segment between the 

predetermined two points and the predetermined coordinate axis. 

For example, as shown in Figure 6 embodiment, by controlling lighting/lights-out 

of the infrared-LED by the processor, it becomes possible to perform the light-emission 

of the strobe light source and the exposure by the imaging means only at a necessary time 
25 interval and at a necessary timing, capable of reducing electric power consumption. 
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If the object includes the reflective body (50, 50A, 100, 1 16), a contrast between 
the object and other images are further enhanced, capable of improving accuracy of the 
detection with an inexpensive configuration. 

In a case of responding to the specific wavelength, the stroboscope includes the 
5 light source (the infrared-LED 42, for example) outputting the light having the specific 
wavelength range, and the imaging means responds only to the specific wavelength range 
by utilizing the infrared filter. Therefore, rendering the light having the wavelength that 
the moving light source and blinking light source (fluorescent lamp, etc.) except for the 
object to be detected do not have the light source of the stroboscope, it is possible to 
10 remove the noise light sources. 

In the embodiment, each of the first means and the second means is a process to be 
processed by a single or a plurality of processors (processors for processing 52 and/or 
63). Thus, the first means and the second means are rendered processes to be processed 
as software of the processor, capable of establishing a system with low prices and with 
1 5 high flexibility. It is noted that it is further desirable that the processes of both of the first 
means and the second means are executed by a single processor. 

Then, the image processing performed by the second means is an entertainment 
process such as a game, etc. 

By utilizing the man-machine interface according to this invention as a 
20 man-machine interface for a personal computer, a workstation, game equipment, 

educational equipment, medical equipment, etc., it is possible to establish the input 
system with low prices and with high degree of precision. 

According to this invention, by digitally analyzing the imaging result of the object 
irradiated by the stroboscope, the information such as the position, the size, the velocity, 
25 the acceleration, the moving path pattern of the object can be used as inputs to the 
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information processing apparatus. 

Furthermore, with simple information processing, it is possible to perform 
detection independent of the noise or the disturbance with high degree of precision, and 
therefore, it becomes possible to use on a system restricted in performance depending on 
5 the condition such as prices, allowable electric power consumption, etc. 

The above described objects and other objects, features, aspects and advantages of 
the present invention will become more apparent from the following detailed description 
of the present invention when taken in conjunction with the accompanying drawings. 



10 BRIEF DESCRIPTION OF THE DRAWINGS 

Figure 1 is an illustrative view showing one example of a strobe image imaged by 
a stroboscope; 

Figure 2 is an illustrative view showing an entire configuration of a golf game 
system of one embodiment of the present invention; 
15 Figure 3 is an illustrative view showing one example of an imaging unit of Figure 

2 embodiment; 

Figure 4 is an illustrative view showing one example of a golf-club-shaped input 
device of Figure 2 embodiment; 

Figure 5 is a block diagram showing Figure 2 embodiment; 
20 Figure 6 is a schematic circuit diagram showing a configuration of fetching pixel 

data into a game processor from an image sensor and an LED driving circuit in Figure 5 
embodiment; 

Figure 7 is a timing chart showing an operation of Figure 6 embodiment; 
Figure 8 is a timing chart showing a part of Figure 7 in an enlarged maimer; 
25 Figure 9 is an illustrative view showing a state or state transition of Figure 2 
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embodiment; 

Figure 10 is a flowchart showing an entire operation of Figure 2 embodiment; 
Figure 1 1 is a flowchart showing a sensor initializing process being one example 
of an initialization process of Figure 10 embodiment; 

Figure 12 is a flowchart showing a command transmitting process in Figure 1 1 ; 
Figure 13 is a flowchart showing a register setting process in Figure 11; 
Figure 14 is a timing chart showing the register setting process shown in Figure 

13; 

Figure 15 is a flowchart showing an operation of a game processor in Figure 2 
embodiment; 

Figure 16 is a flowchart showing a stroboscope imaging process in Figure 15 
embodiment; 

Figure 17 is a flowchart showing a pixel data fetching process in Figure 15 
embodiment; 

Figure 18 is a flowchart showing a notable point extracting process in Figure 15 
embodiment; 

Figure 19 is an illustrative view showing a principle adopting difference data in 
embodiments; 

Figure 20 is a flowchart showing a velocity vector calculating process in Figure 1 5 
embodiment; 

Figure 21 is an illustrative view showing respective coordinates positions in the 
velocity vector calculating process; 

Figure 22. is a flowchart showing a determination process in Figure 15 
embodiment; 

Figure 23 is an illustrative view showing a modified example of the 



golf-club-shaped input device utilized in the golf game system; 

Figure 24 is a flowchart showing the notable point extracting processing operation 
in a case of utilizing the golf-club-shaped input device in Figure 23; 

Figure 25 is a flowchart showing a determination process in Figure 1 5 
embodiment in a case of utilizing the golf-club-shaped input device in Figure 23; 

Figure 26 is an illustrative view showing an angle in the determination processing 
operation in Figure 25; 

Figure 27 is an illustrative view showing an entire configuration of a bowling 
game system of another embodiment of the present invention; 

Figure 28 is a sectional illustrative view showing an intemal configuration of a 
ball-shaped input device shown in Figure 27 at a line XXVIII- XXVIII; 

Figure 29 is an illustrative view shov/ing one example of a game screen to be 
displayed on a television monitor in Figure 27 embodiment; 

Figure 30 is an illustrative view showing one example of a score sheet to be 
displayed on the television monitor in Figure 27 embodiment; 

Figure 31 is a block diagram showing Figure 27 embodiment; 

Figure 32 is a flowchart showing an entire operation of Figure 27 embodiment; 

Figure 33 is a flowchart showing a determination process in Figure 15 
embodiment in the bowling system in Figure 27; 

Figure 34 is an illustrative view showing one example of a glove-shaped input 
device; 

Figure 35 is a flowchart showing a movement detecting processing operation 
shown in Figure 15 embodiment in a case of utilizing the glove-shaped input device in 
Figure 34; and 

Figure 36 is a flowchart showing the pixel data array fetching process in Figure 
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1 5 embodiment. 

BEST MODE FOR PRACTICING THE INVENTION 

With referring to Figure 2, a golf game system 10 of one embodiment of the 
5 present invention includes a game machine 12 and a golf-club-shaped input device 14, 
and the golf-club-shaped input device 14 is swung by a game player over the game 
machine 12. It is noted that the game machine 12 is driven by a direct-current power 
supply such as an AC adapter (not shown) or a battery, for example. The game machine 
12 is further connected to an AV terminal (not shown) of a television monitor (not shown) 

1 0 via an AV cable 1 6. 

The game machine 12 includes a housing 1 8, and on the housing 1 8, a power 
switch 20 is provided, and a direction button 22, a determination key 24 and a cancel key 
26 are also provided. The direction button 22 has respective buttons of four directions (a 
up, down, left and right), and utilized for moving a cursor in order to select a menu or a 

15 game mode on a display screen of a television monitor, for example. The determination 
key 24 is utilized for determining an input to the game machine 12. Furthermore, the 
cancel key 26 is utilized for canceling an input to the game machine 12. 

Inside the housing 1 8 of the game machine 12, an imaging unit 28 shown in Figure 
3 in detail is housed. The imaging unit 28 includes a unit base 30 formed by a plastic 

20 molding, for example, and to the unit base 30, a supporting cylinder 32 is attached. On an 
upper surface of the supporting cylinder 32, an opening 34 whose inside is in a form of 
inverted-conical shaped bugle is formed, an optical system including a concave lens 36 
and a convex lens 38 both formed by a transparent plastic molding, for example, is 
provided inside a cylindrical portion below the opening 34, and beneath the convex lens 

25 38, an image sensor 40 as an imaging device is fixed. Accordingly, the image sensor 40 
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can photograph an image corresponding to a light incident from the opening 34 through 
the lenses 36 and 38. 

The image sensor 40 is a CMOS image sensor of low resolution (for example, 32 
X 32 pixels: gray scale). It is noted that the image sensor 40 may have pixels greater in 
5 number, or may consist of another device such as CCD, or the like. 

Furthermore, the unit base 30 is provided with a plurality of infrared-LEDs (four 
(4) in this embodiment) each having an upper light emitting direction. By the 
infrared-LEDs 42, an infrared light is irradiated within a range defined by the lines 44a 
and 44b above the imaging unit 28 shown in Figure 3. Furthermore, above the unit base 
10 30, an infrared filter (filter for passing through only the infrared ray) is attached so as to 
cover the above-described opening 34. Then, the infrared-LED 42 is as described later 
continuously turned on/off so as to fimction as an infrared stroboscope. It is noted that the 
"stroboscope" is a generic name for intermittently irradiating a moving body. 
Accordingly, the above-described image sensor 40 images a moving body, that is, a 
15 reflective body 50 (see Figure 4) provided on the golf-club-shaped input device 14 in this 
embodiment, which moves within a imaging range shown by lines 44a and 44b. 

The golf-club-shaped input device 14 is entirely formed by a plastic molding, for 
example, and, as shown in Figure 4, includes a club shaft 46 and a club head 48 attached 
at a tip end of it. At the bottom of the club head 48, a circle-shaped reflective body 50 
20 consisting of a retroreflection sheet, for example, is provided. Accordingly, as shown in 
Figure 2, when a game player swings with the club shaft 46 as the input device 44 in his 
hands above the game machine 12 as he generally plays the golf, a light reflected by the 
reflective body 50 at the bottom surface of the head 48 is imaged by the image sensor 40. 
At this time, the infrared-LED 42 intermittently irradiates the infrared light as described 
25 above, and therefore, the reflective body 50 is intermittently imaged as shown in Figure 3. 
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In the golf game system 10 of this embodiment, as described later, by such the strobe 
image processing of the reflective body, a velocity as an input to the game machine 12, 
etc. is calculated. 

With referring to Figure 5, the club-shaped input device 14 is irradiated by a light 
5 emission from the infrared-LED 42, and reflects the infrared light by the reflective body 
50. The reflected light from the reflective body 50 is imaged by the image sensor 40, and 
the image sensor 40 outputs an image signal of the reflective body 50. The analog image 
signal from the image sensor 40 is converted into digital data by an A/D converter (not 
shown) included in a game processor 52. 

10 It is noted that the game processor 52 intermittently blinks the infrared-LED 42 for 

the above-described strobe imaging. 

Although arbitrary kinds of processor can be utilized as such the game processor 
52, a high-speed processor developed by the applicant of the present invention and 
already filed as a patent application is used in this embodiment. This high-speed 

1 5 processor is disclosed in detail in Japanese Patent Laying-open No. HI 0-307790 
[G06F13/36, 15/78] and US Patent No. 6,070,205 corresponding thereto. 

Although not illustrated, the game processor 52 includes various processors such 
as a CPU, a graphics processor, a sound processor, a DMA processor, etc. and also 
includes the above-described A/D converter used for fetching an analog signal, and an 

20 input/output control circuit for receiving an input signal such as a key operation signal 
and an infrared signal, and giving an output signal to external equipment. Therefore, an 
input signal from the operation keys 22-26 is applied to the CPU through the input/output 
control circuit. The CPU executes a required operation according to the input signal, and 
applies the result to the graphics processor, the sound processor, etc. Therefore, the 

25 graphic processor and the sound processor execute an image processing and a sound 
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processing according to the operation result. 

Although not illustrated, the processor 52 is provided with an internal memory, 
and the intemal memory includes a ROM or a RAM (SRAM and/or DRAM). The RAM 
is utilized as a temporary memory, a working memory, a counter, or a register area 
5 (temporary data area), and a flag area. It is noted that the processor 52 is connected with 
a ROM 54 via an external bus. In the ROM 54, a game program described later is set in 
advance. 

The processor 52 processes a digital image signal input from the image sensor 40 
via the AID converter to detect a movement of the golf-club-shaped input device 14, and 

10 executes an arithmetic operation, a graphic processing, a sound processing, etc. to output 
a video signal and an audio signal. The video signal is an image signal for displaying a 
game screen, and the audio signal is a signal for a game music and a sound effect. 
Accordingly, a game screen is displayed on the screen of a television monitor (not 
shown), and a necessary sound (sound effect, game music) is output from its speaker. 

15 Here, with referring to Figure 6 to Figure 8, a configuration for fetching pixel data 

to the game processor 52 from the COMS image sensor 40 is described in detail. As 
shovsoi in Figure 6, the CMOS image sensor 40 in this embodiment is a type of outputting 
a pixel signal as an analog signal, and therefore, the pixel signal is input to an analog input 
port of the game processor 52. The analog input port is connected to an A/D converter 

20 (not shown) within the game processor 52, and thus, the game processor 52 obtains or 
fetches from the A/D converter the pixel signal (pixel data) that has been converted to 
digital data. 

A midpoint of the above-described analog pixel signal is determined by a 
reference voltage applied to a reference voltage terminal Vref of the CMOS image sensor 
25 40. Therefore, in this embodiment, as to the image sensor 40, a reference voltage 
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generating circuit 56 including a resistor voltage-dividing circuit, for example, is 
provided, and from the reference voltage generating circuit 56, a reference voltage having 
a constant magnitude is always applied to the reference voltage terminal Vref. 

Each of digital signals for controlling the CMOS image sensor 40 is applied to the 
I/O port of the game processor 52, or output therefrom. The I/O ports are digital ports 
capable of controlling respective input/outputs, and connected to an input/output control 
circuit (not shown) within the game processor 52. 

More specifically, from the output port of the game processor 52, a reset signal for 
resetting the image sensor 40 is output so as to be applied to the image sensor 40. 
Furthermore, from the image sensor 40, a pixel data strobe signal and a frame status flag 
signal are output, and these signals are applied to the input port of the game processor 52. 
The pixel data strobe signal is a strobe signal as shown in Figure 7 (b) for reading each of 
the pixel signals described above. The frame status flag signal is a flag signal for showing 
a state of the image sensor 40, and defines an exposure period of the image sensor 40 as 
shown in Figure 7 (a). A low level of the frame status signal shown in Figure 7 (a) 
indicates an exposure period, and a high level thereof shown in Figure 7 (a) indicates a 
non-exposure period. 

Furthermore, the game processor 52 outputs from the I/O port a command (or 
command + data) set in a control register (not shown) within the CMOS image sensor 40 
as register data, and outputs a register setting clock in which the high level and the low 
level are repeated and applies it to the image sensor 40. 

It is noted that as the infrared-LED 42, four (4) infrared-LEDs 42a, 42b, 42c, and 
42d connected in parallel with each other are utilized as shown in Figure 6. These four (4) 
infrared-LEDs 42a -42d are arranged such that they irradiate the object (golf-club-shaped 
input device 14) as described above, emits an infrared light in a direction the same as 
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viewpoint direction of the image sensor 40, and surrounds the image sensor 40. It is noted 
that these respective infrared-LEDs 42a-42d are simply called as the infrared-LED 42 
except as especially required by discrimination. The infrared-LED 42 is turned on and is 
extinguished (turned off) by the LED driving circuit 58. The LED driving circuit 58 
receives the above-described frame status flag signal from the image sensor 40, and the 
flag signal is applied to a base of a PNP transistor 68 through a differentiation circuit 60 
consisting of a resistor 62 and a capacitor 64. The PNP transistor 68 is connected with a 
pull-up resistor 66, and the base of the PNP transistor 68 is normally pulled up to a high 
level. Then, when the frame status signal becomes a low level, the low level is input to 
the base through the differentiation circuit 60, and the PNP transistor 68 is turned on only 
during a low level period of the flag signal. 

A collector of the PNP transistor 68 is grounded via resistors 70 and 72. Then, a 
node of collector resistances 70 and 72 is connected to the base of an NPN transistor 74. 
A collector of the NPN transistor 74 is commonly connected to anodes of the respective 
infrared-LEDs 42a-42d. An emitter of the NPN transistor 74 is directly connected to a 
base of another NPN transistor 76. A collector of the NPN transistor 74 is commonly 
connected to cathodes of the respective infrared-LEDs 42a-42d, and the emitter thereof is 
grounded. 

In the LED driving circuit 58, the infrared-LED 42 is tumed on during only a 
period when the LED control signal (corresponding to a second signal) output from the 
I/O port of the game processor 52 is active (high level), and the frame status flag signal 
from the image sensor 40 is a low level. As shown in Figure 7 (a), when the frame status 
flag signal becomes the low level, the PNP transistor 68 is tumed on during the low level 
period (although there is a delay due to the time constant of the differentiation circuit 60 
in reality). Accordingly, when the LED control signal shown in Figure 7 (d) is output 
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from the game processor 52 at the high level, the base of the NPN transistor 74 becomes a 
low level, and then, the transistor 68 is turned off. When the transistor 68 is turned off, 
the transistor 74 is turned on. Accordingly, a current flows a power supply (shown by a 
small white circle in Figure 6) through the respective infrared-LEDs 42a-42d and the 
transistor 76, and in response thereto, the respective infrared-LEDs 42a-42d are turned on 
as shown in Figure 7 (e). 

In the LED driving circuit 58 of this embodiment, the infrared-LED 42 is turned 
on only when the LED control signal shown in Figure 7 (d) is active, and the frame status 
flag signal is low level, and therefore, the infrared-LED 42 is turned on only during an 
exposure period (see Figure 7 (f)) of the image sensor 40. Therefore, according to this 
embodiment, it is possible to reduce useless power consumption. Furthermore, the frame 
status flag signal is coupled by the capacitor 64, and therefore, on assumption that the flag 
signal is suspended with the low level kept due to a hang-up of the image sensor 40 and 
the like, the transistor 68 is surely turned off after a lapse of a predetermined time period, 
and the infrared-LED 42 is surely tumed off after a lapse of the predetermined time 
period. 

Thus, it is possible to change the exposure time period of the image sensor 40 
arbitrarily and freely by changing duration of the frame status signal. 

In addition, by changing the duration or period of the frame status signal and the 
LED control signal, it is possible to arbitrarily and freely set a light-emission period, a 
non light-emission period, a duty cycle of light-emission/non light-emission, etc. of the 
infrared-LED 42, that is, the stroboscope. 

As described previously, when the reflective body 50 of the golf-club-shaped 
input device 14 is irradiated by an infrared light from the infrared-LED 42, the image 
sensor 40 is exposed by the reflected light from the reflective body 50. In response 
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thereto, the pixel signal described above is output from the image sensor 40. More 
specifically, the CMOS image sensor 40 outputs an analog pixel signal shown in Figure 7 
(c) in synchronous with a pixel data strobe shown in Figure 7 (b) during a period when the 
frame status signal shown in the above-described Figure 7 (a) is the high level (a period 
5 when the infrared-LED 42 is not turned on). The game processor 52, while monitoring 
the frame status flag signal and the pixel data strobe, obtains a digital pixel data through 
the A/D converter. 

It is noted that the pixel data (pixel signal) is output in an order of lines such as the 
zero line, the first line, • • - to the thirty-first line as shown in Figure 8 (c). It is noted that 
10 single pixel data at the head of each of lines becomes dummy data. 

Here, with referring to Figure 9 and Figure 10, a brief operation of the golf game 
system 10 of Figure 2 embodiment is described. A game is started by tuming the power 
switch 20 shown in Figure 2 on. It is noted that the game processor 52 shown in Figure 5 
executes an initialization process in a step SI . More specifically, the system and the 
1 5 respective variables are initialized. 

The initialization process in the step S 1 includes a data setting processing with 
respect to the control register contained in the image sensor 40, and is specifically 
executed according to the flowchart shown in Figure 1 1 to Figure 13 and at timing shown 
in Figure 14. 

20 In a first step SI 1 shown in Figure 11, the game processor 52 sets a command 

"CONF" as setting data. It is noted that the command "CONF" is a command for 
informing the image sensor 40 of entering the setting mode for transmitting a command 
from the game processor 52. Then, in a following step SI 3, a command transmitting 
process is executed shown in detail in Figure 12. 

25 In a first step S3 1 of the command transmitting process, the processor 52 sets the 
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setting data (command "CONF" for the step SI 3) to the register data (I/O port), and sets a 
register setting clock (I/O port) to a low level in a next step S3 3. Then, after a wait of a 
predetermined time period in a step S3 5, the register setting clock is set to a high level in 
a step S37. Then, after a wait of a predetermined time period in a step S39, the register 
5 setting clock is set to the low level once again in a step S41 . Thus, as shown in Figure 14, 
by performing the wait of the predetermined time period, the register setting clock is 
changed to the low level, the high level, and the low level one after another, and whereby, 
a transmitting process of the command (command or command + data) is performed. 

In a step SI 5 (Figure 1 1), a pixel mode is set, and an exposure time period is set. 

10 In the case of this embodiment, the image sensor 40 is, as described above, a CMOS 

sensor of 32 X 32, for example, and therefore, "Oh" indicative of being 32 X 32 pixels is 
set to the pixel mode register of the setting address "0". In a next step SI 7, the game 
processor 52 executes a register setting process shown in Figure 13 in detail. 

In a first step S43 of the register setting process, the processor 52 sets a command 

15 "MOV"-i- address as a setting data, and, in a following step S45, executes a command 

transmitting process described above in Figure 12 to transmit it. The processor 52 sets the 
command "LD" + data as the setting data in a succeeding step S47, and executes a 
command transmitting process to transmit it in a following step S49. The processor 52 
sets a command "SET" as the setting data in a step S51 to transmit it in a following step 

20 S53. It is noted that the command "MOV" is a command indicative of transmitting an 
address of the control register, the command "LD" is a command indicative of 
transmitting data, and the command "SET" is a command for actually setting data to its 
address. It is noted that the process is repeated in a case that there are a plurality of 
control registers to be set. 

25 Retuming to Figure 1 1, in a next step SI 9, the setting address is rendered "1" 
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(indicative of low-Nibble address of an exposure time setting register), and low-Nibble 
data "Fh" of "FFh" indicative of a maximum exposure time period is set as data to be set. 
Then, in a step S21, the register setting process in Figure 13 is executed. Similarly, in a 
step S23, the setting address is rendered "2" (indicative of high-Nibble address of an 
5 exposure time setting register), and high-Nibble data "Fh" of "FFh" indicative of the 

maximum exposure time period is set as data to be set, and the register setting process is 
executed in a step S25, Then, in a step S27, a command "RUN" indicative of an end of 
the setting and for starting to output data from the image sensor 40 is set so as to be 
transmitted in a step S29. 

10 Thus, the initialization process is executed in the step SI shovm in Figure 10. It is 

noted that the embodiment shown in Figure 1 1 to Figure 14 can be changed according to 
a specification of the image sensor to be used. 

After the step SI in Figure 10, the game processor 52 updates an image signal so 
as to update an image to be displayed on a monitor (not shown) in a step S2. It is noted 

1 5 that the update of the display image is executed for each frame (television frame or video 
frame). 

Then, the game processor 52 executes a process according to a state. It is noted 
that it is a selection of the game mode that has to be processed first. In the game mode 
selection, a user or a game player selects a game mode such as one-player mode or a 
20 two-player mode by operating the selection key 22 shown in Figure 1 , and sets a difficulty 
level of the game, and so on. 

In an actual golf game, it is necessary to roll a ball by swinging a golf club, and in 
the golf game system 10 in this embodiment, a swinging action of the golf club is 
performed in a real space by use of the golf-club-shaped input device 14. Here, the game 
25 processor 52 executes a swing determining process of the swing action to determine 
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whether or not the swing action is performed in a step S4. Then, when the swing action is 
performed, the ball is flied or rolled in the game screen, and a ball path is calculated in a 
step S5. When the ball is stopped, a score calculating and result determining process is 
executed in a step S6 as a result of the ball path calculating process in the step S5. 

Then, if an interruption occurs in response to a video synchronizing signal, an 
image updating in the step S2 (Figure 1 0) is executed. Furthermore, the sound processing 
in a step S7 is executed at a time of occurrence of a sound interruption, and whereby, a 
game music and a sound effect such as hitting a ball, and so on are output. 

With referring to Figure 15 to Figure 22, a specific entire operation of such the 
golf game system 10 is described. In a first step S55 in Figure 15, the game processor 52 
executes a stroboscope imaging process for detecting a moving position of the 
golf-club-shaped input device 14. 

A detail of the stroboscope imaging process is shown in Figure 16. In a first step 
S67 in Figure 16, the game processor 52 substitutes "1" for the number of times N in the 
number of times register (not shown) at an arbitrary area of the internal memory (not 
shown). In a succeeding step S69, the game processor 52 tums the infrared-LED 42 on 
for the stroboscope imaging. More specifically, the LED control signal shown in Figure 7 
is rendered the high level. Then, in a step S71, a pixel data array fetching process is 
executed. 

In a first step S301 in Figure 36, the game processor 52 sets "-1" to X and sets "0" 
to Y as an element number of the image data array. The pixel data array in this 
embodiment is two-dimensional array of X=0 -31, and Y==0 -31. However, the dummy 
data is output as data of a head pixel in each line as described above, so that "-1" is set as 
an initial value of X. In a succeeding step S3 03, a pixel data fetching process of the 
elements [Y] and [X] shown in Figure 17 is executed. 
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In a first step S83 shown in Figure 1 7, the game processor 52 checks a frame status 
flag signal from the image sensor 40, and in a step S85, it is determined whether or not its 
rising edge (from the low level to the high level) occurs. Then, when it is detected that the 
flag signal rises in the step S85, the game processor 52 instructs the A/D converter inside 
5 thereof to start to convert an input analog pixel signal to digital data in a succeeding step 
S87. Then, in a step S89, it is determined whether or not the pixel strobe from the image 
sensor 40 is present, and in a step S91, it is determined whether or not its rising edge from 
the low level to the high level occurs. 

If "YES" is determined in the step S91, the game processor 52 then determines 

10 whether X=-l or not, that is, whether the head pixel or not in a step S93. As described 
above, the head pixel in each line is set as a dummy pixel, so that if "YES" is determined 
in the step S93, without fetching the pixel data at that time in a next step S95, the element 
number X is incremented in a step S97. 

^ If "NO" is determined in the step S93, the second pixel data onward in the line is 

15 determined, and therefore, the pixel data at that time is fetched in steps S99 and SI 01 so 
as to be stored in a temporary register (not shown). Thereafter, the process returns to a 
step S305 shown in Figure 36. 

In the step S305, the pixel data stored in the temporary register is stored as the 
elements [Y] [X] of the pixel data array. 

20 In a following step S3 09, the X is incremented. If the X is less than 32, the process 

from the step S303 to the step S307 described above is repeatedly executed. If the X is 
equal to 32, that is, the fetch of the pixel data is reached to the end of the line, "-1" is set to 
the X in a following step S3 1 1 and the Y" is incremented in a step S3 1 3 to repeat the pixel 
data fetching process from the head pixel of the next line. 

25 If the Y is 32 in a step S3 15, that is, if the fetch of the pixel data is reached to the 
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end of the pixel data array, the process is returned to a step S73 in Figure 16. 

In the step S 73, the above-described pixel data array is stored as fetched data at 
N-th time lighting in the working area of the internal RAM, for example. 

In a following step S75, the game processor 52 turns the infrared-LED 42 off by 
rendering the LED control signal the low level, and so forth. Thereafter, in a step S76, the 
pixel data array when the infrared-LED 42 is turned off is obtained according to a 
subroutine shown in Figure 1 7 and Figure 35 so as to be stored in the working area of the 
internal RAM in a step S77 similarly to the step S73. 

Then, the number of times register N is incremented in a step S79, and it is 
determined whether or not the number of times N reaches a predetermined value in a step 
S81 . If "YES" in the step S81, the process returns to a step S57 (notable point extracting 
process). It is noted that if "NO", the process is returned to a preceding step S69. 

A detail of the notable point extracting process is shown in Figure 18. In a first 
step SI 03 in Figure 18, the game processor 52 substitutes "1" for the number of times N 
of the number of times register (not shown) in the internal memory (not shown). Then, in 
a following step S105, a difference data array is calculated from differences between the 
fetched data at lighting up and the fetched data at lighting out that are fetched in the step 
S73 and the step S77, respectively. 

That is, in this embodiment, an infrared light is irradiated onto the reflective body 
50 of the golf-club-shaped input device 14, and an image by the reflected infrared light 
incident to the image sensor 40 via the infrared filter is imaged. In a case of 
strobe-imaging the golf-club-shaped input device 14 in a general room environment by 
use of a general light source, in addition to the images of the reflective body, images of all 
things in the room as well as the light source such as the fluorescent light source, an 
incandescent light source, sunlight (window) are imaged on the image sensor 
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(corresponding to the image sensor 40 in this embodiment) as shown in Figure 19 (A). 
Accordingly, in order to process the images in Figure 1 9 (A) to extract only the images of 
the reflective body, a considerably high-speed computer or processor is needed. 
However, in the game apparatus required to be a low cost, such the high-performance 
5 computer is not useable. Thus, it is necessary to perform various processes to reduce a 
load. 

Figure 19 (B) is an image signal when the image signal in Figure 19 (A) is 
subjected to a level discrimination at a certain threshold value. Such the level 
discriminating process can be executed by a dedicated hardware circuit or in respect of 

10 software. However, in any method, when the level discrimination is executed for cutting 
pixel data having light amounts less than a predetermined amount, low luminance images 
except for the reflective body and the light source can be eliminated. In the image shown 
in Figure 19 (B), a processing of the images except for the reflective body and the light 
source in the room are omitted, so that, it is possible to reduce the load of the computer 

15 while the high luminance images including the light source image is still imaged, and 
therefore, it is difficult to discriminate between the reflective body and the light source. 

Here, in this embodiment, as shown in Figure 3, by use of the infrared ray filter 44, 
images except for the images by the infrared ray are not imaged on the image sensor 40. 
Thus, as shown in Figure 19 (C), it is possible to eliminate the images of the fluorescent 

20 lamp source scarcely including the infrared ray. However, the sunlight and the 

incandescent lamp are still included in the image signal. Accordingly, in order to further 
reduce the load, the differences between the pixel data at a time of tuming the infrared 
stroboscope on and the pixel data at a time of tuming it off are calculated. 

More specifically, the differences between the pixel data of the image data at a 

25 time of lighting up shown in Figure 1 9 (C) and the pixel data of the image data at a time of 
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lighting out shown in Figure 1 9 (D) are calculated. Then, the image by the differences is 
obtained as shown in Figure 19 (E). As clearly understood in comparison with Figure 19 
(A), the image by the difference data includes only the images obtained from the 
reflective body of the golf-club-shaped input device 14. Accordingly, even the game 
processor 52 low in performance can obtain a moving path of the reflective body 50, that 
is, the club head 48 (Figure 4) by the swing of the golf-club-shaped input device 14. 

For this reason, in this embodiment, the difference data array shown in Figure 19 
(E), for example, is calculated in a step SI 05 shown in Figure 18. After obtaining the 
difference data array in the step SI 05, coordinates of a pixel having the largest value 
(pixel having the largest luminance) is obtained in a step SI 07, and it is determined 
whether or not the luminance of the pixel at the coordinates exceeds the predetermined 
value in a step S 1 09. 

If "YES" is determined in the step SI 09, it is determined whether or not pixels 
adjacent to the pixel at the coordinates obtained in the step SI 07, and pixels further 
adjacent thereto exceed the above-described predetermined value one after another to 
calculate a diameter 0 [N] of a notable portion (image of the reflective body in the 
embodiment) in a step Sill. The reason why calculating the diameter (or size) of the 
notable portion is it is necessary that the height (Z coordinate) of the reflective body 50 is 
obtained in a step SI 13 and central coordinates of the reflective body are specified in a 
step SI 15. 

As shown in Figure 3, an optical system with a single focus is utilized for the 
imaging unit 28 of this embodiment. Therefore, when the distance between the reflective 
body 50 and the imaging device, that is, the image sensor 40 is coincident with the focus 
of the above-described optical system, the image is reduced in blur and increased in 
diameter. On the contrary thereto, the further the estrangement between the focused 
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distance and the reflective body-image sensor distance is, the smaller the image itself 
becomes. In an example of Figure 19 (E) and Figure 21 described later, the size 
(diameter) of the image is changed depending upon the distance between the reflective 
body and the image sensor. Thus, the distance between the reflective body and the image 
5 sensor, that is, the height of the reflective body 50 (Z coordinate) can be evaluated on the 
basis of the diameter (size) of the image of the reflective body. Although in the golf game 
system of this embodiment, the Z coordinate is not utilized, by utilizing the Z coordinate 
as necessary, a further different game input can be provided. 

Thus, in the step SI 13, the Z coordinate is obtained, and in the step SI 15, central 

10 coordinates (X, Y or X, Y, Z) are saved. 

Then, the N value of the number of times register is incremented in a step SI 1 7, 
and it is determined whether or not the number of times N exceeds a predetermined value 
in a step S 1 1 9. If "YES" is determined in the step S 1 1 9, the process is directly returned to 
the step S59 in Figure 15. However, if "NO" is determined, the process is returned to the 

15 preceding step SI 05 to execute the process in the step SI 05 onward. 

It is noted that if "NO" in the step SI 09, that is, if it is determined the luminance of 
the pixel having the largest luminance does not exceed the predetermined value, it is 
determined whether or not all the predetermined number of data has been searched in a 
following step S121. If "YES" is determined in the step S121, that is, if all the 

20 predetermined number of pixels is searched, determining that the notable point is absent, 
the result of the search is stored in a step SI 23. It is noted that if "NO" is determined in ' 
the step SI 21, the coordinates data of the pixel having the next largest luminance is 
obtained in a step SI 25, and then, the process returns to the step SI 07. 

The notable point extracting process is thus performed in the step S57, and then, a 

25 movement calculating process such as calculating the velocity vector, and so forth is 
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executed in a following step S59. 

Figure 20 shows a detailed embodiment of a velocity vector calculating process as 
one example of the movement calculating process. In a step S127 in Figure 20, "1" is 
substituted for the number of times register N as described above. Then, in a step S 129, 
an N-th velocity vector (VX [N], VY [N]) is calculated by subtracting the (N-l)-th 
notable point coordinates (PX [N-1] PY [N-1]) from the N-th notable point coordinates 
(PX [N], PY [N]: Figure 21) so as to be stored in the intemal memory. 

Figure 2 1 shows an image of the notable area at the first time having a diameter O 
[1], the central coordinates of the notable area at the first time are (PX [1], PY [1]), and 
the central coordinates of the notable area at the second time having a diameter O [2] is 
(PX [2], PY [2]). Similarly, the notable areas third and the fourth time have diameters 0 
[3] and O [4], respectively, and the central coordinates thereof are indicated by (PX [3], 
PY [3]) and (PX [4], PY [4]), respectively. 

In a case that the notable area at the second times is the N-th, the (N-l)-th notable 
area is the notable area at the first time. Thus, in the step S 1 29, the velocity vector in an X 
direction VX [2] is rendered (PX [2] - PX [1]), and the velocity vector in a Y direction VY 
[2] is rendered (PY [2] - PY [1]). It is noted that in a case of N=l, due to the absence of 
the coordinates data of N-1, the velocity vector is calculated by using final resuh data at 
the previous time, or by using the predetermined value in a case of the absence of final 
result data. 

It is noted that in Figure 21 , the change amounts A X and A Y of the notable area 
(reflective body) of each strobe image is also illustrated. Accordingly, if necessary, a 
change or displacement velocity can be calculated with the use of the change amounts. 

After calculating the velocity vector in the step SI 29, the number of times N is 
incremented in a step S131, and it is determined whether or not the N reaches the 
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predetermined value in a following step SI 33. If "NO" in the step SI 33, the process 
returns to the preceding step SI 29 to repeat the process in the step S129. 

After the process in the step S59, a determining process described in detail in 
Figure 22 is executed in a following step S61 . In the determining process, it is determined 
5 whether or not the golf-club-shaped input device 14 is swung. In a step SI 35 in Figure 
22, "1" is substituted for N, and then, in a step SI 37, a velocity V [N] (scalar value) is 
calculated from the velocity vector at the N-th point (VX [N], VY [N]). Then, it is 
determined whether or not the velocity V [N] thus calculated exceeds the first threshold 
value in a following step S139. If "YES" in the step S139, it is directly determined the 

10 swing is performed in a step S141 , and an initial velocity vector of the golf ball on hitting 
is evaluated from the velocity vector at the N-th point. Accordingly, in a case of the golf 
game system in this embodiment, a flying distance can be calculated from the initial 
velocity vector, the wind direction, the strength of the wind, and the geographical data. 
If "NO" in the step SI 39, that is, if the swung speed of the golf club is below the 

15 first threshold value, it is determined whether or not a line segment between the N-th 
point and the (N-l)-th point intersects a predetermined area in a step SI 45. Then, as a 
result of the determination, if "YES" is determined in a step SI 47, it is determined 
whether or not the speed at the N-th point (scalar value) exceeds a second threshold value 
in a step 149. It is noted that the second threshold value is naturally a value smaller than 

20 the first threshold value. 

If "YES" in the step SI 49, the process proceeds to a preceding step SI 41 while if 
"NO", the process proceeds to the step SI 51 just as "NO" in the step SI 47 to increment 
the value N. Then, it is determined whether N is larger than a predetermined value or not, 
and if "NO", the process returns to the preceding step S 137 to repeatedly execute the 

25 steps onward. It is noted that if "NO" is determined in the step S 1 53, that is, in a case that 
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the line segment in the step SI 45 does not intersect the predetermined region, or even if 
the line segment intersects that area, in a case the velocity is smaller than the 
predetermined value, it is determined that the swing is not performed in the end. 

After completion of the determining process in Figure 22, the process retums to a 
5 step S63 shown in Figure 15. In the step S63, a process corresponding to the application 
such as the game and etc. is performed, and in a step S65, it is determined whether or not 
the process is ended (game over in a case of the game), and if "YES", the process is 
ended. 

It is noted that in the above-described embodiment, the circle-shaped reflective 
10 body 50 is provided on the golf-club-shaped input device 14, the initial velocity vector is 
evaluated from the moving path, and regarding that the golf ball is hit at the initial 
velocity, the flying distance of the ball is calculated. That is, the rotation applied to the 
ball is ignored. This is because that it is impossible, if using the circle-shaped reflective 
body, to specify the orientation of the golf-club-shaped input device 14. Here, in a next 
15 embodiment, it is rendered the orientation of the golf-club-shaped input device 14 can 
also be calculated. 

In order to attain this, in the next embodiment, the golf-club-shaped input device 
14 shown in Figure 23 is utilized. In this embodiment, an oval-shaped or 
rectangular-shaped reflective body 5 OA is utilized while the circle-shaped reflectivity 
20 body is utilized for the golf-club-shaped input device shown in the preceding Figure 4. 

Then, after the respective pixel data at a time that the light-emitting diode 42 lights 
up and pixel data at a time the LED 42 lights out are fetched in the steps 85 5 of Figure 15, 
the notable point extracting process shown in Figure 24 is executed. It is noted that the 
steps S157-S163 are the same as the steps SI 03- 109 in Figure 18. 
25 Then, in a following step SI 65, it is determined whether or not pixels adjacent to 
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the pixel of the coordinates obtained in the step SI 61 and a pixel adjacent to the pixel 
exceed the predetermined value one after another to extract all the pixels at the notable 
point (image of the reflective body in this embodiment). Then, in a step S 166, two pixels 
of Pa (Xa, Ya) and Pb (Xb, Yb) that are the farthest from each other are extracted out of 
5 all the pixels of the notable point. The two points, as shown in Figure 26, indicate both 
ends of the rectangular-shaped reflective body 50 A in the direction of the length. There 
are no points that are the farthest from each other except for the both ends of the long side. 

Then, in a step SI 67, middle point coordinates between the two points Pa and Pb 
are stored as the coordinates at N-th point (PX [N], PY [N]) in the memory. Then, in a 

10 step S 1 69, a tilt between the Pa and the Pb is calculated so as to be stored as angle data 6 
[N] as shown in Figure 26. It is noted that the tih d [N] is, as shown in the step SI 69, 
calculated as an arc tangent of (Xa-Xb)/(Ya-Yb). 

Thus, the direction of the golf-club-shaped input device 14 with respect to the 
imaging device is obtained as the angle data 0 [N]. 

1 5 The steps S 1 7 1 -S 1 79 in Figure 24 respectively are the same as the steps 

SI 17-S125 in Figure 18. 

Then, in the next determining process, the respective steps shown in Figure 25 are 
executed. However, steps S 1 8 1 -S 1 89 and steps S 1 9 1 -steps S20 1 are the same as the steps 
S 135-S143 and the steps S145-S155 shown in Figure 22, respectively. 

20 In a step S203 in Figure 25, the angle 6 j (Figure 26) is calculated by the 

coordinates at N-th point and the coordinates at (N-l)-th point. The angle 6 j can be 
calculated by the arc tangent (PY [N] - PY [N-1])/ (PX [N] - PX [N-1]) as shown in the 
step S203. In a step S205, a tih Ok (=6 [N]- 0 j ) of the golf-club-shaped input device 14 
with respect to the direction of the swing of the golf-club-shaped input device 14 is 

25 calculated according to an equation shown in the step S205, and a hook/slice parameter 
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on hitting is calculated from the 6 k. Thus, by obtaining the hook/slice parameter, 
changing a flying direction due to the spin of the ball as well as simply changing the 
flying distance can be added, and therefore, it is possible to provide a more reality and 
interest to the golf game system of this embodiment. 
5 With referring to Figure 27, a bowling game system 78 of another embodiment 

according to this invention includes a game machine 80, and the game machine 80 is 
driven by an AC adaptor or a battery, and connected to an AV terminal (not shown) of a 
television monitor (not shown) through an AV cable 16 similarly to the game machine 12 
in the game system in Figure 2. 

10 The game machine 78 further includes a housing 82, and on the housing 82, a 

power switch 84 is provided, and a direction button 86, a determination key 88, and a 
cancel key 90 are also provided. These buttons or operation keys have the same function 
as corresponding ones in Figure 2. 

A part of the housing 82 of the game machine 80 is partly cut away, and at that 

1 5 place, a movable body 92 is rotatably supported in an elevation direction. On the side 

surface of the movable body 92, the imaging unit 28 described with referring to Figure 3 
is housed, and therefore, the movable body 92 is provided with an image sensor 40 
similarly to the above-described one. Then, an infrared-LED 42 is provided on the side 
surface of the movable body 92 in proximity to the image sensor 40 and in an integrated 

20 manner with the image sensor 40 so as to be moved in the elevation direction, and 
constitutes a stroboscope imaging means. 

The movable body 92 is supported such that it has constant degree of freedom in 
the elevation direction in this embodiment. However, the degree of freedom may be 
provided in a revolution direction in place of the elevation direction, or in addition to the 

25 elevation direction. Specifically, the movable body 92, that is, the image sensor 40 and 
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the infrared-LED 42 are provided changeable in an arbitrary direction. 

It is noted that if a wider angle lens is used as the lens of the image sensor 40 
(concave lens and convex lens in Figure 4), the image sensor 40 needs not to be movable, 
and alternatively fixedly attached to the image sensor 40. 

A ball-shaped input device 94 has holes 94a, 94b, and 94c into which three fingers 
of the user, that is, a thumb, a middle finger, and a ring finger are respectively inserted as 
does in the real bowling game, and is provided with a wider hole 94d into which a child 
can insert any one or a plurality fingers except for the thumb. Then, a strap 96 is 
provided, and therefore, the game player equips it in his arms (upper arm or front arm) in 
order to assure the safety of the player. That is, the ball-shaped input device 94 is coupled 
to his arm by the strap 96, and therefore, even if the game player releases the ball-shaped 
input device 94 from his fingers as does in the real bowling game, it is possible to prevent 
accidents of the ball-shaped input device 94 being flown away, being hit against the 
player himself or others, and so forth. 

Furthermore, the ball-shaped input device 94 in this embodiment forms the 
housing of the ball-shaped input device 94 by connecting transparent or translucent 
hemisphere outer shells 98A and 98B to each other by bosses, and inside the hemisphere 
outer shells 98A and 98B, hemisphere inner shells 1 OOA and 1 OOB that are similarly 
coupled to each other by bosses are fixed. Then, a reflective sheet is pasted on the surface 
of the respective hemisphere inner shells 1 OOA and lOOB to form a reflective body. That 
is, the inner shells become the reflective body. Accordingly, the reference numeral 100 is 
assigned to the reflective body in this embodiment. 

Furthermore, in the bowling game system 78 in this embodiment, the movement 
of the ball-shaped input device 94 is detected by the stroboscope, and whereby, a position 
of a bowling ball 104 is controlled on a game screen 102 as shown in Figure 29. The 
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game screen 102 is displayed as a perspective image viewed from a viewpoint of the user 
or the player. That is, on the game screen 1 02, a bowling lane 1 06 and pins 1 08 arranged 
in a depth direction are displayed. On the game screen, the bowling ball 104 is moved on 
the lane 1 06, and in correspondence to a reached position and intensity, the pins are 
knocked down as do in the real bowling game. It is noted that if from directly before the 
bowling ball 104 hit the pins 108, an enlarged image of the pins portion is displayed on 
the window (not shown) at the center of the screen, for example, so much presence can be 
provided to the players. 

It is noted that every time that the player finish throwing the ball, the game screen 
102 shown in Figure 29 and a score sheet shown in Figure 30 are switchably displayed. 
Then, in a case that a plurality of game players play the game, the scores of the respective 
players are simultaneously displayed. Figure 30 example shows an example in which 
four game players simultaneously go in the bowling game. 

In the bowling game system 78, when the player performs an throwing action in a 
real space by use of the ball-shaped input device 94, the game processor 52 (Figure 31) 
intermittently turns the infrared-LED 42 on, and intermittently detects the positions of the 
ball-shaped input device 94 by analyzing and processing the images of the CMOS image 
sensor 40 at each lighting-up time and lighting-out time. Then, in correspondence to the 
positions (coordinates) of the ball-shaped input device 94, the movement of the bowling 
ball 1 04 is controlled to knock down zero or more pins. 

With referring to Figure 31, the ball-shaped input device 94 as described above is 
irradiated by a light emission from the infrared-LED 42, and reflects the infrared ray by 
the reflective body 100. The reflected light from the reflective body 100 is imaged by the 
CMOS image sensor 40, and whereby, an image signal of the reflective body 100 is 
output from the CMOS image sensor 40. It is noted that the other part is the same as the 
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golf game system 10 shown in Figure 5. 

Herein, with referring to Figure 32, an operation of the bowling game system 78 in 
this embodiment is schematically described. After a power switch 84 is turned on to start 
a game shown in Figure 27, the game processor 52 shown in Figure 3 1 first executes an 
5 initializing process in a step S 1 . More specifically, systems and respective variables are 
initialized. It is noted that a detailed method of the initialization is described above. 

Then, after the step SI in Figure 32, the game processor 52 updates an image 
signal to update an image to be displayed on the monitor 20 in a step S2. It is noted that 
the update of the display image is executed every one frame (television firame or video 
10 fi-ame). 

Then, the game processor 52 executes a process depending on the state (status). It 
is noted that a process to be executed first is a game mode selection. In the game mode 
selection, the user or the game player selects the game mode such as an one-player mode, 
a two-player mode, etc. by operating the selection key 86 shown in Figure 27, and sets a 

1 5 difficulty level of the game in a step S3 in Figure 32. 

Although in the real bowling game, it is necessary to throw the ball on the lane, in 
the bowling game system 10 in this embodiment, a throwing action is performed by use of 
the ball-shaped input device 94 as described above. Here, the game processor 52 
executes a throwing determining process to determine whether or not the throwing action 

20 is performed in a step S4. Then, if the throwing action is performed, when the ball 1 04 
moves on the lane 106 (the both in Figure 30), the ball path is calculated and a hitting 
determining process of the ball 104 against the pins 108 (Figure 30) is executed in a step 
S5. Then, when the ball 104 reaches the end of the lane 106, a score calculating and result 
determining process is executed in a step S6 as the result of the pin hitting determining 

25 process in the step 85. 
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It is noted that in the bowling game system 78 in this embodiment, by imaging the 
reflective body 100 with the stroboscope, the game input is performed, and this is the 
same as the preceding embodiment. Accordingly, this embodiment is different from the 
preceding embodiment in only the determining processing in the step S61 in Figure 15. 

The throwing determining process step is shown in detail in Figure 33, and an N 
value is set to "1" in a first step S207. Then,, in a step S209, it is determined whether or 
not a Y component of the velocity vector (component in up and down directions) VY [N] 
(Figure 21) exceeds a predetermined value. If "YES" in the step S209, the process 
proceeds to a step S21 1 to increment a counter formed in an internal memory, for 
example. Then, in a next step S2 13, it is determined whether or not a count value of the 
counter is a predetermined constant C (for example, "3"). If "YES", it is determined that 
a throwing action is performed in a step S215. In a succeeding step S217, the X 
coordinate PX [N] at the N-th point is rendered as the X coordinate at the throwing 
position, and in a step S 219, an initial velocity of the ball on throwing is evaluated from 
the velocity vector at N-th point (VX [N], VY [N]). Then, according to the initial 
velocity, regarding that the bowling ball is thrown, the game processing is executed. 

It is noted that if "NO" in the step S209, the above-described counter is reset in a 
step S221. Accordingly, the throwing action is not determined until the Y component of 
the velocity vector exceeds the predetermined value successive three times, for example. 
Thus, it is possible to prevent an undesired action of the game player from being reflected 
on the game. 

After the step S221 , the N value is incremented in a step S223, and it is determined 
whether or not the N value reaches a predetermined value in a step S225. If "YES" is 
determined, it is determined that the throwing is not performed in a step S227. Then, the 
process returns to the step S63 in Figure 15 similarly to the step S219 onward. 
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Figure 34 shows another embodiment of an input device utilizing a stroboscope. 
The input device 1 12 of this embodiment is a glove-shaped input device. The 
glove-shaped input device 112 includes gloves 1 14L and 1 14R to be attached to both 
right and left hands, and reflective bodies 11 6L and 1 16R in the form of the reflective 
sheet are provided at predetermined positions (the tip end portion) of the gloves 1 141 and 
1 14R. These reflective bodies 1 16L and 1 16R are respectively formed as a part of the 
gloves 1 14L and 1 14R, and alternatively may be pasted on the gloves 1 14L and 1 14R. 

Then, in order to apply an input signal, the user puts on the gloves 11 4L and 1 14R, 
and moves both hands over the imaging unit 28 (Figure 3) of the game machine 80 shown 
in Figure 27, for example. Then, according to the step S55 in Figure 15 described above 
or Figure 16, both of the reflective bodies 1 16L and 1 16R are irradiated by the 
infrared-LED 42, or not irradiated, and imaged by the image sensor 40. Then, according 
to the step S57 in Figure 15, that is, Figure 18, a notable point (two reflective bodies 116 
and thus two notable points in this embodiment) is extracted. Thereafter, an action 
calculating or detecting process is executed by applying the process in the step S59 in 
Figure 15. It is noted that in a case of utilizing the glove-shaped input device 1 12 in 
Figure 34, a modification is applied to the step S59 as shown in Figure 35. 

In a flowchart shown in Figure 35, a moving average is detected or calculated to 
obtain an input. Describing in detail, in a first step S207, a "predetermined value -M" is 
set to the number-of-times register N. Next, in a step S209, "0" is set into the variables E 
X and E Y. 

In a step S21 1, coordinates (PX [N], PY [N]): Figure 21) atN-th time are obtained. 
It is noted that in a case of N < 1, coordinates data is obtained from coordinate 
information of the last time. In a succeeding step S213, the coordinates obtained in the 
step S21 1 are respectively added to the variables Z X and Z Y initialized in the step S209 
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to update the variables Z X and Z Y. This process is repeated until it is detected in a step 
S217 that the number of times N incremented in the step S215 reaches the predetermined 
value. Accordingly, the game processor 52 stores, at this time, the variables E X and E Y 
to which M sets of coordinates are added. Then, in a step S21 9, each of the variables £ X 
and Z Y is divided by the number M to calculate the moving average (AX, AY). By use 
of the moving average AX, AY, the game processor 52 changes positions of moving 
objects that can be operated by the player on the game screen. 

It is noted that the present invention can be embodied or modified except for the 
above-described embodiments. 

For example, in the embodiment shown in Figure 2, the input is performed by the 
use of the golf-club-shaped input device 14. However, by utilizing the similar system, a 
number of modifications are conceivable including a baseball game apparatus by use of a 
bat-shaped input device for baseball and/or a ball-shaped input device for baseball, a table 
tennis game by use of a racket-shaped input device for table teimis, a tennis game by use 
of a racket-shaped input device for termis, etc. 

Furthermore, a modification of soccer game apparatus is also conceivable, in 
which by attaching an input device provided with a reflective body around the part of the 
lower leg or the ankle, a position, velocity, a pattern of a moving path of the leg of the 
player are used as an input signal. 

In addition, as to the embodiment utilizing the glove-shaped input device shown in 
Figure 34, a moving average value calculated in the flowchart in Figure 35 is utilized as 
an input signal. Here, modifications including a boxing game apparatus and a dancing 
game apparatus are conceivable, in which a position, a moving velocity, a moving path of 
the glove-shaped input device are calculated to be used as an input signal. Furthermore, 
in these modified examples, in place of the glove-shaped input device, a wristband shaped 



input device wrapped around the wrist secures the same effect as the glove-shaped input 
device. 

A further modification including a dancing game apparatus utilizing the arms and 
the legs is conceivable, in which the above described input device wrapped around the leg 
5 is used in combination with the glove-shaped input device shown in Figure 34 or the 
wristband shaped input device. 

In addition, other modifications including a sword battle game apparatus are 
conceivable, in which like the golf-club-shaped input device 14 shown in Figure 23, the 
long-thin shaped reflective body 50A is pasted on the input device in the form of a sword, 
10 and an angle, a position, a moving velocity, a moving path of the sword are used as an 
input device. 

According to this invention, an object is imaged by use of the stroboscope and the 
imaging means, and on the basis of the difference between the plurality of image signals 
at a time of lighting up and an image signal at a time of lighting out, a part or all of the 
15 information of the position, the size, the velocity, the acceleration, and the moving path 
pattern are calculated. The information processing apparatus and an entertainment 
apparatus execute the information processing, the game, and other entertainment 
processing by use of the information. 

It is noted that although all the information processing is executed by a single 
20 game processor in the above-described embodiment, entire processing can be shared with 
the use of two or more processors or computers. 

Although the present invention has been described and illustrated in detail, it is 
clearly understood that the same is by way of illustration and example only and is not to 
be taken by way of limitation, the spirit and scope of the present invention being limited 
25 only by the terms of the appended claims. 
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